Interactive Task Estimation From Unlabelled Teaching Signals

At home, workplaces or schools, an increasing amount of intelligent robotic systems are starting to be able to help us in our daily life (windows or vacuum cleaners, self-driving cars) [1] and in flexible manufacturing systems [2]. A key feature in these new domains is the close interaction between people and robots. In particular, such robotic systems need to be teachable by non-technical users, i.e. programmable for new tasks in novel environments through intuitive, flexible and personalized interactions. Specifically, the challenge we address in this work is how a robot learning a new task can be instructed by a human using its own preferred teaching signals, where the mapping between these signals and their meaning is unknown to the robot. For example, different users may use different languages, words, interjections, gestures or even brain signals to mean “good” or “bad”. While research on robot learning from human interaction has flourished in the last ten years [3], most work has focused on how to extract statistical task models from human teachers following a fixed pre-defined teaching protocol. Thus, a usual assumption is that the learner and the teacher share a mutual understanding of the meaning of each others’ signals. The question of how a robot can learn to interpret personalized and potentially noisy teaching signals, i.e. learn instruction models, has been much less explored. In a preliminary work [4], we presented a computational approach addressing this problem by considering a finite space of teaching signals in simulation while bootstrapping the system with known symbols. Later [5], we released the need for bootstrapping and allow the teacher to use any kind of signal that can be represented as a fixed length feature vector, which is better suited for Human-Robot Interaction (HRI) scenario. Interestingly, a similar approach [6] has been developed in the Brain Computer Interfaces (BCI) community. Braincomputer interfaces provide a communication channel between humans and machines using only brain activity. Recent works have shown that it is possible to decode information related to human error processing, namely the error-related potentials [7] appearing for instance when the device action does not match the user’s expectations. This potential has been used as feedback information to solve sequential tasks [8][9]. In practice, BCI solves the meaning problem using an openloop calibration procedure to train a decoder in a supervised manner. In this abstract, we address the problem of removing the need for calibration. We propose an algorithm that simultaneously calibrates the feedback decoder and executes in closed loop a sequential task only known by the user. Our method exploits task constraints, namely optimal policies, to hallucinate the meaning of the teaching signals and select the task with the lowest expected error. We report results from a HRI scenario and a BCI online experiments. The results show that the proposed method is able to learn good feedback models while solving the task efficiently without any calibration. In next section, we will provide an intuition on how the algorithm works without providing mathematical details. Then results from real world experiments will be briefly described, followed by the conclusion.